Agent robot control system, agent robot system, agent robot control method, and storage medium

ABSTRACT

[Problem] To provide an agent robot control system, agent robot system, agent robot control method, and recording medium, which are capable of assisting in optimal shopping in real storefronts and online storefronts. [Solution] Provided is an agent robot control system, comprising an acquisition unit which acquires a purchasing master list which shows a user&#39;s purchase merchandise candidates, and a control unit which generates a purchasing execution list for recommending, from the purchasing master list, purchases for select merchandise to be performed at real storefronts and for recommending purchases for other merchandise to be performed at online storefronts.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is continuation application of U.S. patent applicationSer. No. 16/314523, filed on Dec. 31, 2018, which is a U.S. NationalPhase of International Patent Application No. PCT/JP2017/015510 filed onApr. 17, 2017, which claims priority benefit of Japanese PatentApplication No. JP 2016-138933 filed in the Japan Patent Office on Jul.13, 2016. Each of the above-referenced applications is herebyincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an agent robot control system, anagent robot system, an agent robot control method, and a storage medium.

BACKGROUND ART

In recent years, the society has been aging, and in daily life, goingout for shopping involves a great deal of labor for elderly people. Inaddition, many of two-job families have difficulties in takingsufficient time for going shopping. In response to such difficulties,nowadays, a delivery service provided from a supermarket and internetshopping have become widespread.

In addition, in recent years, a shopping assist technology that uses arobot has been proposed.

For example, Patent Literature 1 described below discloses a pet robotthat carries a baggage. In addition, Patent Literature 2 described belowdiscloses a humanoid robot that guides shops, explains about commercialproducts, and carries goods. In addition, Patent Literature 3 describedbelow discloses a mascot robot that is attached to a shopping cart in aspecific shop, and assists shopping by informing a shopper of reportinformation such as bargain sale. In addition, Patent Literature 4described below discloses a shopping support system in which, whencommercial products are given to robots disposed in a plurality oflocations in a shop, the commercial products are conveyed to a checkoutcounter, and the commercial products can be collectively received beforecheckout.

CITATION LIST Patent Literature

Patent Literature 1: JP 2010-046742A

Patent Literature 2: JP 2014-188597A

Patent Literature 3: JP 2013-151045A

Patent Literature 4: JP 2008-87892A

DISCLOSURE OF INVENTION Technical Problem

Nevertheless, if a user tries to buy a necessary product in a shopoffering the lowest price, when the product is necessary, the user needsto manually perform internet search to decide what product is to bebought in which shop, and this involves labors and time. In addition,depending on a commercial product, the commercial product can besometimes obtained earlier at lower price by purchasing the commercialproduct in a nearby physical shop, rather than purchasing the commercialproduct in an internet shop (online shop), but further labors arerequired for seeking for an optimum way of buying by comprehensivelyresearching these.

In addition, none of Patent Literatures described above mentionsrecommending optimality as to what commercial product is to be purchasedin which of an online shop and a physical shop, among commercialproducts necessary for the user.

In view of the foregoing, the present disclosure proposes an agent robotcontrol system, an agent robot system, an agent robot control method,and a storage medium that can support optimum shopping in a physicalshop and an online shop.

Solution to Problem

According to the present disclosure, there is proposed an agent robotcontrol system including: an acquisition section configured to acquire apurchase master list indicating a candidate of a purchase commercialproduct of a user; and a control section configured to generate apurchase execution list for recommending purchase in a physical shop fora partial commercial product of the purchase master list, andrecommending purchase in an online shop for another commercial product.

According to the present disclosure, there is proposed an agent robotsystem including: a server including a first communication sectionconfigured to acquire a purchase master list indicating a candidate of apurchase commercial product of a user, and a first control sectionconfigured to generate a purchase execution list for recommendingpurchase in a physical shop for a partial commercial product of thepurchase master list, and recommending purchase in an online shop foranother commercial product, and perform control so as to transmit thepurchase execution list to an agent robot via the first communicationsection; and an agent robot including a second communication sectionconfigured to receive the purchase execution list, an output sectionconfigured to output information to a user, and a control sectionconfigured to control the output section to output, to the user,information recommending purchase in a physical shop for the partialcommercial product, and recommending purchase in an online shop for theother commercial product, on the basis of the purchase execution list.

According to the present disclosure, there is proposed an agent robotcontrol method including: by a processor, acquiring a purchase masterlist indicating a candidate of a purchase commercial product of a user;and generating a purchase execution list for recommending purchase in aphysical shop for a partial commercial product of the purchase masterlist, and recommending purchase in an online shop for another commercialproduct.

According to the present disclosure, there is proposed a storage mediumhaving an agent robot control program stored thereon, the agent robotcontrol program causing a computer to function as: an acquisitionsection configured to acquire a purchase master list indicating acandidate of a purchase commercial product of a user; and a controlsection configured to generate a purchase execution list forrecommending purchase in a physical shop for a partial commercialproduct of the purchase master list, and recommending purchase in anonline shop for another commercial product.

Advantageous Effects of Invention

As described above, according to the present disclosure, it becomespossible to support optimum shopping in a physical shop and an onlineshop.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram describing an overview of an information processingsystem according to an embodiment of the present disclosure.

FIG. 2 is a diagram illustrating an example of an overall configurationof the information processing system according to an embodiment of thepresent disclosure.

FIG. 3 is a block diagram illustrating an example of a configuration ofa pet robot according to an embodiment of the present disclosure.

FIG. 4 is a block diagram illustrating functional configuration examplesof a control section and a storage section according to an embodiment ofthe present disclosure.

FIG. 5 is a diagram illustrating an example of a purchase master listaccording to the present embodiment.

FIG. 6 is a diagram illustrating an example of an online shop listaccording to the present embodiment.

FIG. 7 is a diagram illustrating an example of a physical shop listaccording to the present embodiment.

FIG. 8 is a diagram describing a positional relationship between aphysical shop and a home according to the present embodiment.

FIG. 9 is a flow chart illustrating operation processing of purchaseshop recommendation according to a first embodiment of the presentdisclosure.

FIG. 10 is a diagram illustrating an example of a purchase completionmanipulation screen according to the first embodiment of the presentdisclosure.

FIG. 11 is a flow chart illustrating generation processing of an onlineshop item list according to the first embodiment of the presentdisclosure.

FIG. 12 is a diagram illustrating an example of an online shop item listaccording to the first embodiment of the present disclosure.

FIG. 13 is a flow chart illustrating generation processing of a physicalshop item list according to the first embodiment of the presentdisclosure.

FIG. 14 is a diagram describing object recognition from a captured imageof an inside of a shop according to the first embodiment of the presentdisclosure.

FIG. 15 is a diagram illustrating an example of a physical shop itemlist according to the first embodiment of the present disclosure.

FIG. 16 is a diagram describing an overview of purchase execution listgeneration according to the first embodiment of the present disclosure.

FIG. 17 is a flow chart illustrating generation processing of a purchaseexecution list according to the first embodiment of the presentdisclosure.

FIG. 18 is a diagram illustrating an example of a purchase executionlist according to the first embodiment of the present disclosure.

FIG. 19 is a flow chart illustrating in-refrigerator item learningprocessing according to a second embodiment of the present disclosure.

FIG. 20 is a diagram illustrating an example of a currentin-refrigerator item list and a previous in-refrigerator item listaccording to the second embodiment of the present disclosure.

FIG. 21 is a diagram illustrating an example of a difference listaccording to the second embodiment of the present disclosure.

FIG. 22 is a diagram illustrating an example of a purchase cycleprobability list according to the second embodiment of the presentdisclosure.

FIG. 23 is a flow chart illustrating recommendation processing of apurchase item according to the second embodiment of the presentdisclosure.

FIG. 24 is a flow chart illustrating operation processing according to athird embodiment of the present disclosure.

FIG. 25 is a flow chart illustrating operation processing according to afourth embodiment of the present disclosure.

FIG. 26 is a block diagram illustrating an example of a configuration ofa server according to an embodiment of the present disclosure.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. Notethat, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

-   In addition, the description will be given in the following order.

1. Overview of Information Processing System According to Embodiment ofPresent Disclosure 2. Configuration 2-1. Overall Configuration 2-2.Configuration of Pet Robot 1 3. Operation Processing 3-1. FirstEmbodiment 3-2. Second Embodiment 3-3. Third Embodiment 3-4. FourthEmbodiment 4. Supplement 5. Conclusion «1. Overview of InformationProcessing System According to Embodiment of Present Disclosure»

First of all, an overview of an information processing system accordingto an embodiment of the present disclosure will be described. FIG. 1 isa diagram describing an overview of an information processing systemaccording to the present embodiment. As illustrated in FIG. 1, theinformation processing system according to the present embodiment isimplemented by a pet robot 1 (information processing device) having anentertainment property that is formed to have an external shapeimitating an animal such as a dog. The pet robot 1 is an example of anagent that can express animal-like gesture by causing eyes, legs, andthe like to autonomously operate.

(Background)

As described above, if a user tries to buy a necessary product in a shopoffering the lowest price, when the product is necessary, the user needsto manually perform internet search to decide what product is to bebought in which shop, and this involves labors and time. Users havingcomputer literacy may not be suffered so much, but for elderly peopleand users who have not computer literacy, this can be a difficult andstressful work. In addition, depending on a commercial product, thecommercial product can be sometimes obtained earlier at lower price bypurchasing the commercial product in a nearby physical shop, rather thanpurchasing the commercial product in an internet shop (online shop), butfurther labors are required for seeking for an optimum way of buying bycomprehensively researching these.

In view of the foregoing, in the present disclosure, optimum shopping ina physical shop and an online shop can be supported. In the coming agingsociety and two-job society, it is very useful to provide assistance insuch a manner that, in daily life, a product demanded by a user can bepurchased at a lower price at an optimum timing.

Specifically, on the basis of a predetermined purchase list, forexample, the information processing system according to the presentdisclosure recommends an optimum purchase shop (e.g. a shop where acommercial product can be purchased at the lowest price) in view of theprices of each commercial product in an online shop and a physical shop,and dates (delivery dates) on which each commercial product is deliveredto the user from an online shop and a physical shop. In addition, on thebasis of daily behavior of the user, the information processing systemaccording to the present disclosure can also learn what product is to bepurchased at which timing, and recommend a product to be purchased(perform automatic generation of a purchase list), at a time point atwhich the learning has progressed.

In addition, if the learning further progresses and accuracy increases,on the basis of an automatically-generated purchase list, a robot canalso automatically purchase a product at an optimum shop (purchase aproduct by connecting to an online shop, and purchase a product byautonomously moving to a physical shop). For example, in the case of thepet robot 1 that is autonomously movable as illustrated in FIG. 1, it ispossible to move to a physical shop to purchase a commercial product,and carry the commercial product to a home of the user. In the exampleillustrated in FIG. 1, the pet robot 1 having a dog shape or the likecan carry a commercial product to the home of the user by drawing atrolley 6 and asking a shop staff to put the commercial product into thetrolley 6, for example. Note that the shape of the pet robot 1 is notlimited to this, and for example, the pet robot 1 may be of a walkingtype, a wheel type, or a drone type that enables autonomous movement. Inthe case of a bipedal walking humanoid agent robot, an operation ofseizing a commercial product and putting the commercial product into ashopping cart by using an arm and fingers of an upper half of the bodyis also enabled.

Aside from voice, interaction with a shop staff and the user can beperformed by a display section 21 (e.g. touch panel display) provided onthe pet robot 1.

Note that the pet robot 1 according to the present embodiment is notlimited to a robot that can perform autonomous movement, and may have ashape of a small animal such as a bird, and may be of ashoulder-perching type or a hand-perching type. In this case, it isimpossible to autonomously move to a physical shop, but it is possibleto support shopping by navigating the user to a physical shop by voiceor the like. In addition, it is possible to automatically performpurchase processing in an online shop by connecting to the internet.

An overview of the information processing system according to thepresent embodiment has been described above. Note that, in the presentembodiment, the pet robot 1 is used an example of an informationprocessing device, but the present embodiment is not limited to this,and for example, a virtual substance displayed on a display terminalsuch as a tablet terminal or a smartphone, that is to say, a tiny robotcreated by a software program may be used.

Subsequently, an overall configuration of the information processingsystem according to the present embodiment and a configuration of thepet robot 1 will be specifically described with reference to FIGS. 2 to8.

«2. Configuration»

<2-1. Overall Configuration>

FIG. 2 is a diagram illustrating an example of an overall configurationof the information processing system according to the presentembodiment. As illustrated in FIG. 2, the information processing systemaccording to the present embodiment includes the pet robot 1 and aserver 3. The pet robot 1 and the server 3 are connected via a network2, and can perform transmission and reception of data.

The server 3 includes a physical shop database (DB) 32 and an onlineshop DB 33. In the physical shop DB 32 and the online shop DB 33, piecesof shop information (shop name, classification of the type of shop,access information, operating hours, etc.) of online shops 5 (5A to 5C,and so on) and physical shops 4 (4A, 4B, and so on) are stored. Theserver 3 can construct the databases by collecting pieces of shopinformation in advance from the online shops 5 and the physical shops 4connected via the network 2, for example.

<2-2. Configuration of Pet Robot 1>

Subsequently, a configuration of the pet robot 1 according to thepresent embodiment will be described with reference to FIG. 3. FIG. 3 isa block diagram illustrating an example of a configuration of the petrobot 1 according to the present embodiment. As illustrated in FIG. 3,the pet robot 1 includes a control section 10, a communication section11, a position information acquisition section 14, a camera 15, a voiceinput section 16, a biosensor 17, a drive section 18, a touch sensor 19,a storage section 20, the display section 21, a voice output section 22,an acceleration sensor 23, and an angular velocity sensor 24.

The control section 10 functions as an arithmetic processing unit and acontrol device, and controls the entire operations performed inside thepet robot 1, in accordance with various types of programs. The controlsection 10 is implemented by an electronic circuit such as a CentralProcessing Unit (CPU) or a microprocessor, for example. In addition, thecontrol section 10 may include a Read Only Memory (ROM) that storesprograms, calculation parameters, and the like that are to be used, anda Random Access Memory (RAM) that temporarily stores appropriatelyvarying parameters and the like.

In addition, the control section 10 according to the present embodimentcan perform autonomous control of automatically operating in accordancewith various types of information obtained from the communicationsection 11, the position information acquisition section 14, the camera15, the voice input section 16, the biosensor 17, the touch sensor 19,the acceleration sensor 23, the angular velocity sensor 24, or the like.

The communication section 11 is a communication module for performingtransmission and reception of data with another device. For example, thecommunication section 11 performs transmission and reception of data byconnecting with various types of servers via the network 2, and performstransmission and reception of data by directly connecting with aperipheral device (not illustrated) using Bluetooth (registeredtrademark) or Wi-Fi (registered trademark).

The position information acquisition section 14 has a function ofdetecting a current position of the pet robot 1 on the basis of anacquired signal from the outside. Specifically, for example, theposition information acquisition section 14 is implemented by a GlobalPositioning System (GPS) positioning section, detects a position wherethe pet robot 1 exists, by receiving radio waves from GPS satellites,and outputs the detected position information to the control section 10.The position information may be accumulated in the storage section 20 asa position log of the pet robot 1. In addition, aside from the GPS, theposition information acquisition section 14 may be configured to detecta position by Wi-Fi, Bluetooth, or transmission and reception with amobile phone, a PHS, a smartphone, or the like, near fieldcommunication, or the like, for example.

The camera 15 includes a lens system including an imaging lens, adiaphragm, a zoom lens, a focus lens, and the like, a drive system thatcauses the lens system to perform a focus operation and a zoomoperation, a solid-state image sensor array that photoelectricallyconverts imaging light obtained by the lens system, to generate animaging signal, and the like. The solid-state image sensor array may beimplemented by a Charge Coupled Device (CCD) sensor array or aComplementary Metal Oxide Semiconductor (CMOS) sensor array, forexample.

The voice input section 16 collects voice of the user and surroundingenvironmental sound, and outputs a voice signal to the control section10. The voice input section 16 is implemented by a microphone, amicrophone amplifier section that performs amplification processing of avoice signal obtained by the microphone, and an A/D converter thatconverts the voice signal into digital, and outputs the voice signal tothe control section 10.

The biosensor 17 detects biological information of the user that hastouched the pet robot 1. The biological information of the user is, forexample, body temperature, an amount of perspiration, beats, afingerprint, a palm print, blood pressure, brain waves, and the like.

The drive section 18 is a functional module for implementing a degree offreedom at each joint of the pet robot 1, and includes a plurality ofdrive units provided for respective axes such as roll, pitch, and yaw ineach joint. Each drive unit is formed by a combination of a motor thatperforms a rotational operation around a predetermined axis, an encoderthat detects a rotational position of the motor, and a driver thatadaptively controls a rotational position and a rotational speed of themotor on the basis of an output of the encoder.

The touch sensor 19 detects pressure received from the user by aphysical approach such as “stroking” and “striking”. Note that, in placeof or in addition to the touch sensor 19, the pet robot 1 may include apressure sensor.

The storage section 20 stores programs and the like for the controlsection 10 executing various types of processing. In addition, thestorage section 20 includes a storage device including a storage medium,a recording device that records data into the storage medium, a readingdevice that reads out data from the storage medium, a deletion devicethat deletes data recorded in the storage medium, and the like.

The display section 21 displays various types of screens such as amanipulation screen and a menu screen. In addition, the display section21 according to the present embodiment displays a purchase list, arecommended commercial product, a guide of an optimum shop, and thelike. The display section 21 may be a display device such as a LiquidCrystal Display (LCD) or an organic Electroluminescence (EL) display,for example.

The voice output section 22 is implemented by a speaker and an amplifiercircuit for the speaker. In addition, the voice output section 22outputs voice such as chirp sound.

The acceleration sensor 23 and the angular velocity sensor 24 detectorientation of the pet robot 1 and acceleration of a motion.

The configuration of the pet robot 1 according to the present embodimenthas been specifically described above. Note that the configuration ofthe pet robot 1 according to the present embodiment is not limited tothe example illustrated in FIG. 2, and for example, the pet robot 1 mayinclude other sensors such as a geomagnetic sensor, an ultrasonicsensor, a proximity sensor, an illumination sensor, a temperaturesensor, and a barometric sensor. In addition, as a distance sensor formeasuring a distance to an object positioned in front, the pet robot 1may use the camera 15 described above, or additionally include adistance sensor that uses a method such as infrared light.

In addition, as illustrated in FIG. 2, for example, the pet robot 1 mayinclude a body portion unit 200, leg portion units 201 respectivelycoupled to the anterior left and right and posterior left and right ofthe body portion unit 200, and a head portion unit 202 and a tailportion unit 203 that are respectively coupled to an anterior endportion and a posterior end portion of the body portion unit 200. In thebody portion unit 200, the control section 10 formed by connecting aCentral Processing Unit (CPU), a Dynamic Random Access Memory (DRAM), aflash Read Only Memory (ROM), a Personal Computer (PC) card interfacecircuit, and a signal processing circuit to each other via an internalbus, and a battery serving as a power source of the pet robot 1 arestored. In addition, in the body portion unit 200, the communicationsection 11, the position information acquisition section 14, thebiosensor 17, the storage section 20, the acceleration sensor 23, theangular velocity sensor 24, and the like are also stored. In addition,the display section 21 is installed on a breast portion of the bodyportion unit 200. An installation location of the display section 21 isnot limited to this, and for example, may be a back portion of the bodyportion unit 200, a forehead portion of the head portion unit 202, orthe like.

In addition, in the head portion unit 202, the camera 15 for capturingan image of an external situation, the touch sensor 19 for detectingpressure received from the user by a physical approach such as“stroking” and “striking”, the voice input section 16 for collectingexternal sound, the voice output section 22 for outputting voice such aschirp sound, a distance sensor (not illustrated) for measuring adistance to an object positioned in front, and the like are respectivelydisposed at predetermined positions. The camera 15 may be provided at aposition corresponding to an “eye” of the pet robot 1. In addition, thecamera 15 may be disposed in a forehead portion of the head portion unit202, and a Light Emitting Diode (LED) (not illustrated) may be disposedat a position corresponding to an “eye”. In addition, the voice inputsection 16 may be disposed at a position corresponding to an “ear” ofthe head portion unit 202, and the voice output section 22 may bedisposed at a position corresponding to a “mouth” of the head portionunit 202.

Furthermore, actuators and potentiometers are provided in numberscorresponding to the number of degrees of freedom, at joint portions ofthe respective leg portion units 201, coupling portions of therespective leg portion units 201 and the body portion unit 200, acoupling portion of the head portion unit 202 and the body portion unit200, and the like. For example, an actuator includes a servomotor as aconfiguration. By the driving of the servomotor, the leg portion units201 are controlled to transition to a target attitude or operation.

Regarding a specific configuration example of the pet robot 1 describedabove, for example, JP 2002-157596A is referred to. The entire contentsof JP 2002-157596A are hereby incorporated by reference.

In addition, the configuration of the pet robot 1 described withreference to FIG. 3 can also be applied to a case where the pet robot 1is a tiny robot. Specifically, a display terminal that displays a tinyrobot includes configurations corresponding to the control section 10,the communication section 11, the position information acquisitionsection 14, the camera 15, the voice input section 16, the biosensor 17,the touch sensor 19, the storage section 20, the display section 21, andthe voice output section 22 that have been described above. The tinyrobot is displayed in a display section, and can perform interactionwith the user.

(Functional Configuration)

Subsequently, functional configurations of the control section 10 andthe storage section 20 will be described with reference to FIG. 4. FIG.4 is a block diagram illustrating functional configuration examples ofthe control section 10 and the storage section 20 according to anembodiment of the present disclosure. In the drawing, as functions ofthe control section 10 of the pet robot 1, a purchase master listacquisition section 101, a shop list generation section 102, a purchaseexecution list generation section 103, a movement path decision section104, a drive control section 105, a display control section 106, a voiceoutput control section 107, a user manipulation recognition section 108,a voice recognition section 109, an online purchase processing section110, an object recognition section 111, and a learning section 112 areillustrated. In addition, as functions of the storage section 20 of thepet robot 1, a purchase master list DB 210, an online shop list DB 220,a physical shop list DB 230, an online shop item list DB 240, a physicalshop item list DB 250, and a purchase execution list DB 260 areillustrated. Hereinafter, each constituent element will be furtherdescribed.

The purchase master list acquisition section 101 acquires a purchasemaster list indicating purchase candidate items (commercial products),and stores the purchase master list into the purchase master list DB210. The purchase master list may be arbitrarily input by the user froma creation screen of a purchase master list, may be automaticallygenerated by the purchase master list acquisition section 101, or may bereceived via the communication section 11. As an input performed by theuser, a voice input (recognized by the voice recognition section 109), amanipulation input performed via a touch panel display (recognized bythe user manipulation recognition section 108), an input performed byanother communication terminal and received by the communication section11, or the like is assumed. In addition, the purchase master listacquisition section 101 can also generate a purchase master list on thebasis of a learning result of the learning section 112. In addition, thepurchase master list acquisition section 101 outputs the acquiredpurchase master list to the purchase execution list generation section103.

Here, FIG. 5 illustrates an example of a purchase master list. Asillustrated in the drawing, a purchase master list 211 includes datasuch as a master number, a product name or an article, a category, thenumber or volume, and a required delivery date d1. The “master number”is a serial number (master number) allocated to an item to be purchased,and corresponds to a master number in each piece of data to be describedlater. In addition, the “product name or article” is a commercialproduct name or an article of an item. The “category” is not anessential item, but is input in a case where it is difficult to identifyan item only by a product name or an article, or there are the sameproduct names in different categories. The “number or volume” is arequired number or volume of an item. The “required delivery date d1” isinformation regarding a delivery date on which a corresponding item isto be delivered to the user. Based on the information, determination canbe performed as to whether an item is urgently required by the user,whether delivery may be after several days, or the like.

On the basis of the purchase master list, the shop list generationsection 102 performs search as to at which price and on which deliverydate each item can be purchased in each online shop or each physicalshop, and generates an online shop item list and a physical shop itemlist. The generation of each item list will be described later withreference to FIGS. 11 to 15. In addition, the generated item lists maybe respectively stored in the online shop item list DB 240 and thephysical shop item list DB 250.

Note that a shop list is used in the generation of an item list. Theshop list may be prestored in the online shop list DB 220 or thephysical shop list DB 230, or may be generated on the basis ofinformation acquired in each case from the physical shop DB 32 and theonline shop DB 33 of the server 3 on a network.

Here, data included in the respective shop lists will be described withreference to FIGS. 6 and 7. FIG. 6 is a diagram illustrating an exampleof an online shop list according to the present embodiment. Asillustrated in the drawing, an online shop list 221 includes data suchas a shop name (ID), classification of the type of shop, URL or accessinformation, shop holiday, opening hour, and closing hour. The “shopname (ID)” is an online shop name or a unique ID for identifying acorresponding shop. The “type” is not an essential item, but ifinformation that enables estimation of a commercial product categoryhandled by a shop, such as a food shop, a supermarket, a home appliancestore, and a general store is input, item search efficiency in thegeneration of a purchase execution list to be described later can beenhanced. The “URL or access information” is a URL indicating an accessdestination address or information regarding an API that is forperforming item search, acquisition of latest information, and the like,by accessing a corresponding online shop. Note that, in the case of aservice requiring the use of a dedicated API for performing search, anAPI access key, an authentication code, and the like are included in theaccess information. The “shop holiday” is input in a case where anonline shop has a shop holiday. In the “opening hour” and the “closinghour”, operating hours of an online shop are input. Normally, in thecase of an online shop, purchase processing is assumed to be receivedthroughout 24 hours, but in the case of a service in which a shopdirectly performs delivery, as in an internet supermarket, an orderreception time of the current day is determined in some cases for thesake of delivery time. Thus, such time information is also included.

FIG. 7 is a diagram illustrating an example of a physical shop list 231according to the present embodiment. As illustrated in the drawing, thephysical shop list 231 includes data such as a shop name (ID),classification of the type of shop, a position, shop holiday, openinghour, and closing hour. The “shop name (ID)”, the “type”, the “shopholiday”, the “opening hour”, and the “closing hour” are items similarto those in the online shop list illustrated in FIG. 6. Thus, thedescription will be omitted here. The “position” is position informationsuch as latitude and longitude of a corresponding physical shop. Notethat, because the user or the pet robot 1 needs to actually move tophysical shops included in the physical shop list 231, for itempurchase, for example, as illustrated in FIG. 8, a list of physicalshops including physical shops 4A, 4B, and 4C that are located within aconstant distance from a home of the user, among physical shops 4A to4D, is used (e.g. sorting of a physical shop list may be performed bythe shop list generation section 102 by inputting position informationof the home, or the user may select shops). Note that limitation onphysical shops is not limited to a distance from the home, and forexample, physical shops near a nearest station of a work place, physicalshops near a road frequently used by the user in commuting or the like,and the like may be included.

On the basis of a physical shop item list and an online shop item list,the purchase execution list generation section 103 generates a purchaseexecution list indicating a physical shop or an online shop that isoptimum for the purchase of each item. The details of purchase executionlist generation will be described later with reference to FIGS. 16 to18. The purchase execution list may be edited in accordance with a usermanipulation. For example, in the generated purchase execution list, anitem of which purchase has been input by a user manipulation is shiftedto a “purchase completion” state. In addition, the generated purchaseexecution list is stored into the purchase execution list DB 260. In thepurchase execution list DB 260, a past history of a purchase executionlist is stored together with time and date information.

The movement path decision section 104 decides a movement order ofphysical shops registered in the purchase execution list. For example,on the basis of current position information of the pet robot 1, andpieces of position information of all the physical shops included in thepurchase execution list, the movement path decision section 104 decides,as a movement path, a route in which all the physical shops can bevisited with the shortest distance. In addition, a determinationalgorithm of a movement path is not specifically limited. For example,an existing algorithm such as a solution of a traveling salesman problemmay be used. In addition, in accordance with content of a commercialproduct to be purchased in each shop, the movement path decision section104 may modify a movement path so as to purchase heavy goods later, forexample.

The drive control section 105 controls a movement of the pet robot 1 soas to visit physical shops in accordance with the movement path.Specifically, for example, the drive control section 105 recognizescurrent position information of the pet robot 1 using the positioninformation acquisition section 14, and controls the drive section 18 tocause the pet robot 1 to autonomously move to a target location. Inaddition, the drive control section 105 also controls a movement to anexhibition location of a target commercial product in a physical shop,and a movement to a checkout counter. In a shop, for example, movementcontrol is performed by recognizing a surrounding situation inaccordance with a result recognized by the object recognition section111 on the basis of a captured image obtained by capturing an image of aperiphery of the pet robot 1 by the camera 15.

The display control section 106 performs information display controlwith respect to the display section 21. For example, in a case where thepet robot 1 arrives at a target physical shop, the display controlsection 106 displays a commercial product to be purchased in the shop,on the display section 21, and presents the commercial product to theuser or a shop staff. In addition, the display control section 106displays a screen for presenting, to the user, a purchase item list, arecommended commercial product that is based on a learning result, andin addition, a movement path, and the like.

The voice output control section 107 controls a voice output from thevoice output section 22. For example, in a case where the pet robot 1does not include the display section 21, when the pet robot 1 arrives ata target physical shop, the voice output control section 107 outputs, byvoice, the description about a commercial product to be purchased in theshop. In addition, the voice output control section 107 may notify, byvoice, the user of a purchase item list, a recommended commercialproduct that is based on a learning result, and in addition, a movementpath, and the like.

The user manipulation recognition section 108 recognizes a manipulationinput from the user. For example, a user manipulation input from a touchpanel display in which the touch sensor 19 is integrated with thedisplay section 21 is recognized.

The voice recognition section 109 recognizes a voice input from theuser. For example, user speech voice collected by the voice inputsection 16 such as a microphone is analyzed, and speech content isrecognized.

The online purchase processing section 110 accesses an online shopregistered in the purchase execution list, and performs purchaseprocessing of a corresponding item.

The object recognition section 111 analyzes a captured image captured bythe camera 15, and performs object recognition of the periphery of thepet robot 1. An algorithm of the object recognition is not specificallylimited, and an existing algorithm may be used. A recognition result isoutput to the learning section 112 and the drive control section 105.

The learning section 112 learns an item necessary for the user, and atiming (delivery date) of item purchase. A learning result is output tothe purchase master list acquisition section 101. The details of thelearning processing according to the present embodiment will bedescribed later with reference to FIGS. 19 to 23.

The functional configurations of the control section 10 and the storagesection 20 according to the present embodiment have been specificallydescribed above. Subsequently, operation processing of the informationprocessing system according to the present embodiment will be describedwith reference to FIGS. 9 to 25.

«3. Operation Processing»

3-1. First Embodiment

First of all, operation processing of purchase shop recommendationaccording to the first embodiment will be described with reference toFIGS. 9 to 18.

FIG. 9 is a flow chart illustrating operation processing of purchaseshop recommendation according to the first embodiment of the presentdisclosure. As illustrated in FIG. 9, first of all, the control section10 of the pet robot 1 acquires, using the purchase master listacquisition section 101, a purchase master list being a list of items tobe purchased by the user (step S103). Here, for example, a purchasemaster list is arbitrarily input by the user from a creation screen of apurchase master list that is displayed on the display section 21.Alternatively, a purchase master list is input by the user by voice.

Next, on the basis of the purchase master list, the shop list generationsection 102 searches for items in online shops, and generates an onlineshop item list that summarizes which item can be purchased at what pricein each shop (step S106). The details will be described later withreference to FIGS. 11 and 12.

Subsequently, on the basis of the purchase master list, the shop listgeneration section 102 searches for items in physical shops, andgenerates a physical shop item list that summarizes which item can bepurchased at what price in each shop (step S109). The details will bedescribed later with reference to FIGS. 13 to 15.

Next, on the basis of the physical shop item list and the online shopitem list, the purchase execution list generation section 103 generatesa purchase execution list that decides which commercial product is to befinally purchased in which shop (step S112). The details of the purchaseexecution list generation will be described later with reference toFIGS. 16 to 18. The generated purchase execution list is stored into thepurchase execution list DB 260 together with time and date information,for example.

Subsequently, the movement path decision section 104 decides a movementpath of physical shops registered in the purchase execution list (stepS115).

Next, on the basis of the decided movement path, the drive controlsection 105 moves the pet robot 1 to a physical shop (step S118). Atthis time, the display control section 106 and the voice output controlsection 107 may navigate the user to the physical shop.

Subsequently, when the pet robot 1 arrives at a target physical shop,the display control section 106 displays, on the display section 21, anitem to be purchased in the shop (purchase list) (step S121). In a casewhere the pet robot 1 does not include the display section 21, thedescription of a purchase item is performed by voice by the voice outputcontrol section 107. In a case where the pet robot 1 is accompanied bythe user, purchase of an item is performed by the user. Alternatively,the pet robot 1 performs purchase of an item by presenting a purchaselist to a shop staff of the shop. In a case where the pet robot 1 isprovided with a carrying device of items (the trolley 6 illustrated inFIG. 1, etc.), a purchased item is stored into the carrying device.

Next, when the purchase is completed, a purchase completion manipulationis input by the user or a shop staff (step S124). The purchasecompletion manipulation is recognized by the user manipulationrecognition section 108. Here, FIG. 10 illustrates an example of apurchase completion manipulation screen. In the drawing, a list of itemsto be purchased in a physical shop where the pet robot 1 currentlyexists, check boxes, and a purchase completion button are displayed onthe display section 21. The user checks a check box of a purchased item,and taps the purchase completion button. Note that the purchasecompletion manipulation is not limited to the example illustrated inFIG. 10, and for example, the pet robot 1 may read a sales receiptobtained at the time of purchase, using the camera 15, and recognizepurchase completion of an item by Optical Character Reader (OCR).

Subsequently, in the purchase execution list, the purchase executionlist generation section 103 shifts a status of an item of which thepurchase has been completed, to the “purchase completion” state (stepS127). A specific example of a purchase execution list will be describedlater with reference to FIG. 18. The purchase execution list includes apurchase shop and a purchase status (status) of each item. Asillustrated in FIG. 18, at the time of purchase execution listgeneration, the fields of “purchase status” are all blank, and forexample, when a purchase completion manipulation is performed by theuser, the “purchase completion” state is registered.

Steps S118 to S127 described above are repeated until the pet robot 1moves to all the physical shops registered in the purchase executionlist (step S130). Note that the control section 10 searches for anoptimum shop again for a target item that has not been purchased in aphysical shop, and has a status being left blank in the purchaseexecution list (at this time, excluding the physical shop where the itemhas not been purchased), and in a case where an optimum physical shop isfound, updates the purchase execution list, and controls the pet robot 1so as to move in accordance with a movement path incorporating thephysical shop.

Subsequently, the control section 10 of the pet robot 1 displays, on thedisplay section 21, items in the purchase execution list that areplanned to be purchased in an online shop, and presents the items to theuser (step S133). Note that the pet robot 1 may speak, by voice, theitems planned to be purchased. At this time, the purchase execution listgeneration section 103 may switch the purchase of an item of whichpurchase has not been completed in a physical shop, to purchase in anoptimum online shop. In addition, the control section 10 searches for anoptimum shop again for a target item that has not been purchased in aphysical shop, and has a status being left blank in the purchaseexecution list (at this time, excluding the physical shop where the itemhas not been purchased), and in a case where an optimum online shop isfound, updates the purchase execution list, and presents the item aswell, to the user as an item planned to be purchased in an online shop.

Subsequently, in a case where a deletion instruction for a presentedpurchase execution list is issued from the user by a manipulation inputor a voice input, the purchase execution list generation section 103deletes an instructed item from the purchase execution list (step S136).

Next, the online purchase processing section 110 performs processing ofcollectively purchasing items left in the purchase execution list, foreach online shop (step S139).

Subsequently, in the purchase execution list, the purchase executionlist generation section 103 shifts a status of an item of which purchasehas been completed, to the “purchase completion” state (step S142).

Then, the control section 10 of the pet robot 1 updates the purchasemaster list in accordance with a change in status of the purchaseexecution list (step S145). For example, the control section 10 deletesan item of which purchase has been completed, from the purchase masterlist.

As described above, the pet robot 1 according to the present embodimentuses, as a purchase master list, a list of items input by the user, anddecides an optimum purchase shop of each item. In the case of a physicalshop, the pet robot 1 guides the user to the physical shop, and presentsan item to be purchased. In addition, in the case of an online shop, thepet robot 1 performs purchase processing by automatically accessing acorresponding online shop.

In this manner, it is possible to support purchase of an item desired bythe user, in an optimum shop, and convenience of the pet robot 1 indaily life is further enhanced.

(Generation of Online Shop Item List)

Subsequently, the generation of an online shop item list that isperformed by the shop list generation section 102 in step S106 describedabove will be specifically described with reference to FIG. 11.

FIG. 11 is a flow chart illustrating generation processing of an onlineshop item list according to the present embodiment. As illustrated inFIG. 11, first of all, the shop list generation section 102 acquirescurrent time and date (step S153), and generates, as an online shoplist, shops that can receive an order at the current time and date (stepS156).

Next, the shop list generation section 102 acquires one piece of shopinformation from the online shop list (step S159), and in addition,acquires one unprocessed item from the purchase master list (step S162).

Subsequently, the shop list generation section 102 acquires informationregarding the number of stocks, a price, and a delivery date (d2) in anonline shop of the acquired item (step S165). The shop list generationsection 102 accesses a Website of a corresponding online shop on thebasis of a URL included in the shop information, for example, andsearches for the number of stocks of a target item.

Subsequently, it is checked whether or not the number of stocks is equalto or larger than the necessary number (step S168). In a case where thenumber of stocks is insufficient (step S168/No), the shop listgeneration section 102 returns to step S162, and searches the onlineshop for the next item.

On the other hand, in a case where the number of stocks is sufficient(step S168/Yes), the shop list generation section 102 determines whetheror not a delivery date satisfies time requirement (step S171).Specifically, the shop list generation section 102 determines whether ornot the delivery date d2 is earlier than the delivery date d1 on whichan item is required.

Next, in a case where the delivery date satisfies time requirement (stepS171/Yes), the shop list generation section 102 registers prices in anitem list of each online shop (step S174).

Subsequently, steps S162 to S171 described above are repeated until allthe items (purchase items) included in the purchase master list areprocessed (step S174).

Then, until all the purchase items are processed in all the online shopsincluded in the online shop list, steps S159 to S174 described above arerepeated (step S177). Note that, considering the type of an online shop,the shop list generation section 102 can also preferentially process anitem estimated to be sold in the online shop, to enhance efficient.

In this manner, an online shop item list is generated. Here, FIG. 12illustrates an example of an online shop item list. An online shop itemlist 241 illustrated in FIG. 12 includes price information of eachpurchase item of each online shop. An item for which the number ofstocks is insufficient and an item for which a delivery date is too lateare not registered in the online shop item list. As illustrated in FIG.12, it can be seen that the price of the same item (same master number)varies depending on the shop.

(Generation of Physical Shop Item List)

Next, the generation of a physical shop item list that is performed bythe shop list generation section 102 in step S109 described above willbe specifically described with reference to FIG. 13.

FIG. 13 is a flow chart illustrating generation processing of a physicalshop item list according to the present embodiment. As illustrated inFIG. 13, first of all, the shop list generation section 102 searches thephysical shop DB 32 for information regarding physical shops that arecurrently in business and located within a constant distance from thehome, and generates the information as a physical shop list (step S203).Considering a travel time to a physical shop, the shop list generationsection 102 may search for physical shops that are to be in business ata time point of an hour later from the current time, for example.

Next, the movement path decision section 104 decides a movement path ofphysical shops on the basis of the physical shop list (step S206).

Subsequently, the drive control section 105 moves the pet robot 1 to aphysical shop in accordance with the decided movement path (step S209).

Next, the drive control section 105 moves the pet robot 1 into a shop,and the camera 15 captures an image of the inside of the shop (stepS212). For example, the camera 15 captures an image of a commercialproduct exhibited in the shop.

Subsequently, the object recognition section 111 performs objectrecognition from a captured image obtained by capturing the image of theinside of the shop (step S215). For example, as illustrated in FIG. 14,the object recognition section 111 recognizes objects 151 a and 151 bfrom a captured image 151 obtained by capturing an image of anexhibition rack in the shop.

Next, in a case where there is still an unprocessed item in the purchasemaster list (step S218/Yes), the object recognition section 111 alsorecognizes characters and numbers in a certain range around theabove-described recognized object (step S221). For example, in a casewhere the object recognition section 111 recognizes the objects 151 aand 151 b from the captured image 151 as illustrated in FIG. 14, theobject recognition section 111 further recognizes character strings 152a and 152 b, and numbers (price tags) 153 a and 153 b.

Subsequently, the shop list generation section 102 determines whether ornot content of a recognized character string, a recognized object, andan unprocessed item match (step S224). For example, in a case whereprocessing is performed on “xx morning (milk)” as an unprocessed item,the shop list generation section 102 compares the objects 151 a and 151b and the character strings 152 a and 152 b that have been recognizedfrom the captured image 151 as illustrated in FIG. 14, with “xx morning(milk)”. In this case, the object 151 a and the character string 152 aand “xx morning (milk)” match, and it is confirmed that the unprocesseditem exists in the shop.

Next, in a case where these match (step S224/Yes), the shop listgeneration section 102 registers a recognized price of a correspondingitem in a physical shop item list (step S227). The price of the item isacquired on the basis of the number 153 a recognized from the capturedimage 151, for example. Normally, a price tag is placed on a commercialproduct itself or placed near the commercial product. Thus, on the basisof an object recognition result obtained from a captured image, the shoplist generation section 102 regards a set of a number and a currencymark (¥, $, Yen, etc.) arranged near a character string or an objectthat indicates the name of a commercial product, as a price, andacquires the price of the commercial product.

Each time image capturing of the inside of the shop is performed, theshop list generation section 102 repeats the processing illustrated insteps S221 to S227 described above, for all the items included in thepurchase master list (step S218).

Subsequently, when an unprocessed item becomes nonexistent in thepurchase master list (step S218/No), in a case where the inside of theshop has not been entirely shot yet (step S230/No), the pet robot 1moves to a location in the shop that has not been shot (step S233), andrepeats processing subsequent to step S212 described above. Note thatthe pet robot 1 may perform the above-described object recognition byreceiving a shot image of the inside of the shop from another pet robot1, or may preferentially move to a location where there is a highpossibility that an item included in the purchase master list isexhibited, with reference to a goods classification signboard in theshop.

On the other hand, in a case where the inside of the shop has beenentirely shot (step S233/Yes), if the pet robot 1 has not moved to allthe physical shops (step S236/No), the pet robot 1 moves to the nextphysical shop (step S209). Steps S209 to S230 described above arerepeated until the pet robot 1 moves to all the physical shops includedin the physical shop list (step S236).

In this manner, a physical shop item list is generated. Here, FIG. 15illustrates an example of a physical shop item list. A physical shopitem list 251 illustrated in FIG. 15 includes price information of eachpurchase item of each physical shop. An item not found in a physicalshop is not registered in the physical shop item list. As illustrated inFIG. 15, it can be seen that the price of the same item (same masternumber) varies depending on the shop.

(Generation of Purchase Execution List)

Next, the generation of a purchase execution list performed by the shoplist generation section 102 that is illustrated in step S112 describedabove will be specifically described with reference to FIGS. 16 to 18.

FIG. 16 is a diagram describing an overview of purchase execution listgeneration according to the present embodiment. As illustrated in FIG.16, on the basis of the purchase master list 211 and the physical shoplist 231, the stock and price of each item in each physical shop areacquired, and the physical shop list 231 is generated. In addition, onthe basis of the purchase master list 211 and the online shop list 221,the stock and price of each item in each online shop are acquired, andthe online shop item list 241 is generated. Then, these item lists areintegrated, and a purchase execution list 261 indicating an optimum shopfor the purchase of each item is generated. The generation processing ofthe purchase execution list 261 will be specifically described withreference to FIG. 17.

FIG. 17 is a flow chart illustrating generation processing of a purchaseexecution list according to the present embodiment. As illustrated inFIG. 17, first of all, the purchase execution list generation section103 selects one unprocessed item from the purchase master list (stepS253).

Next, the purchase execution list generation section 103 searches anonline shop item list and a physical shop item list for a shop where theabove-described selected item can be purchased at the lowest price (stepS256). At this time, in a case where shipping cost is required, theshipping cost can be considered.

Subsequently, the purchase execution list generation section 103registers, in the purchase execution list, a lowest price purchasedestination shop (ID) of the corresponding item that has been retrieved(step S259).

Next, the processing illustrated in steps S253 to S259 described aboveis repeated for all items in the purchase master list (step S262).

In this manner, a purchase execution list is generated. Here, FIG. 18illustrates an example of a purchase execution list. The purchaseexecution list 261 illustrated in FIG. 18 includes items of a masternumber, a corresponding optimum purchase shop (ID), and a purchasestatus (status). It becomes possible for the pet robot 1 to recommend anoptimum shop of each item to the user, on the basis of the purchaseexecution list 261. In addition, when the purchase of an item has beencompleted, a state in a purchase status is changed to “purchasecompletion”.

3-2. Second Embodiment

In the first embodiment described above, the user arbitrarily inputs apurchase master list, but the present disclosure is not limited to this,and the pet robot 1 can also automatically generate a purchase masterlist by learning an item necessary for the user. Hereinafter, an exampleof purchase master list automatic generation will be described withreference to FIGS. 19 to 23. Here, for example, a case where content(items) in a refrigerator, a shelf, or the like of the user in a home isalways recognized, consumed goods and supplied goods (replenished good)are detected, a purchase timing is learned, and the user is promoted topurchase an item so as not to forget to purchase (recommendation ofitem) will be described. Note that, here, an increase and a decrease(replenishment history) of items in the refrigerator are automaticallyrecognized and a purchase probability is calculated, but the presentembodiment is not limited to the inside of the refrigerator, and apurchase probability may be calculated on the basis of a purchasehistory (or replenishment history) of general commercial products of theuser.

FIG. 19 is a flow chart illustrating in-refrigerator item learningprocessing according to the second embodiment. As illustrated in FIG.19, first of all, it is determined whether or not the pet robot 1 is ina state of staying in the home (step S303). Because the pet robot 1 canperform an autonomous movement, a case where the pet robot 1 goes outfollowing the user is also assumed. Thus, for example, the controlsection 10 of the pet robot 1 determines whether or not the pet robot 1stays in the home of the user, on the basis of current positioninformation acquired by the position information acquisition section 14.

Next, in a case where the pet robot 1 stays in the home (step S303/Yes),when a refrigerator opens (step S306/Yes), the pet robot 1 captures animage of the inside of the refrigerator using the camera 15 (step S309).For example, the pet robot 1 continuously performs monitoring ofsurrounding environmental sound using the voice input section 16, andmonitoring of a surrounding situation using the camera 15, and when theuser approaches the refrigerator, the pet robot 1 autonomously moves tothe vicinity of the refrigerator following the user, and when the useropens the refrigerator, the pet robot 1 shoots the inside of therefrigerator using the camera 15. In addition, opening-closing of therefrigerator may be detected by an opening-closing sensor, and whennotification is received, the pet robot 1 may move to the vicinity ofthe refrigerator and shoot the inside of the refrigerator using thecamera 15. In addition, the pet robot 1 may receive a captured image ofthe inside of the refrigerator that has been shot by a camera installedin the refrigerator or near the refrigerator.

Subsequently, using the object recognition section 111, the controlsection 10 analyzes the captured image and recognizes goods in therefrigerator, and generates a current in-refrigerator item list (stepS312).

Next, the control section 10 compares the generated currentin-refrigerator item list with a previous in-refrigerator item list, andgenerates a difference list (step S315). Here, FIG. 20 illustratesexamples of the current in-refrigerator item list and the previousin-refrigerator item list. As illustrated in FIG. 20, in a currentin-refrigerator item list 271, information regarding product names orarticles of items currently stored in the refrigerator, and the numberof the items is indicated. In addition, in a previous in-refrigeratoritem list 272, information regarding product names or articles ofin-refrigerator items recognized by the pet robot 1 when the user hasopened the refrigerator the last time, and the number of the items isindicated. The control section 10 compares these item lists andgenerates a difference list. Here, FIG. 21 illustrates an example of adifference list according to the present embodiment. In a differencelist 273 illustrated in FIG. 21, difference information from the lasttime of each item is indicated. In the example illustrated in thedrawing, it is indicated that “product name: XX morning”, “article:tomato ketchup”, and “article: carrot” each decrease by one, “article:egg” increases by six, “article: tofu” and “article: shortcake” eachincrease by one, and “product name: blessing of orange of XX” has nodifference.

Subsequently, the control section 10 learns the difference list as aninput (step S318). Specifically, the control section 10 stores thegenerated difference list and the current in-refrigerator item list intothe storage section 20 together with current time and date.

By repeatedly performing the learning processing described above, thecontrol section 10 can output a purchase cycle probability of each item.In addition, by also learning event days such as birthday and Christmas,it becomes possible to obtain a purchase probability associated with anevent. On the basis of the learned difference list, the control section10 regards a day on which in-refrigerator items increase, as a purchaseday (or identifies a purchase day on the basis of the purchase executionlist), calculates a timing (replenishment day) at which each item is tobe purchased, and a purchase cycle is predicted. Here, FIG. 22illustrates an example of a purchase cycle probability list according tothe present embodiment. As illustrated in FIG. 22, in a purchase cycleprobability list 274, a purchase cycle prediction probability of eachitem is indicated. In the example illustrated in the drawing, forexample, it can be seen that a purchase probability in one day of the“product name: XX morning” is 0.13%, a purchase probability in two daysis 0.58%, and a purchase probability in three days is 0.22%.

Subsequently, the case of performing recommendation of a purchase itemthat is based on the above-described learning will be described withreference to FIG. 23. FIG. 23 is a flow chart illustratingrecommendation processing of a purchase item according to the presentembodiment. The recommendation processing illustrated in FIG. 23 may beperiodically performed, may be performed when a purchase master list isacquired, or may be performed when the user opens the refrigerator, andan in-refrigerator item list is updated by the pet robot 1.

As illustrated in FIG. 23, first of all, the control section 10acquires, from the purchase execution list DB 260, a purchase executionlist including a past history (step S333). A limit value of a pasthistory to be acquired may be set to about one month to one year, forexample.

Next, the control section 10 acquires a lastly-generated in-refrigeratoritem list from the storage section 20 (step S336).

Subsequently, the control section 10 obtains the number of days elapsedfrom purchased time and date of an item stored in the refrigerator (stepS339). Specifically, the control section 10 retroactively searches arecent history of a purchase execution list for each item included inthe in-refrigerator item list, and identifies a purchase day. Bycomparing the purchase day and current date, the control section 10recognizes how many days elapsed from when a corresponding item has beenpurchased.

Subsequently, the control section 10 acquires a purchase probabilityfrom the purchase cycle probability list (step S342). Specifically, thecontrol section 10 acquires, from the purchase cycle probability list274 as illustrated in FIG. 22, a purchase probability in the number ofdays elapsed after purchase of each item.

Next, in a case where a purchase probability is equal to or larger thana threshold value (first threshold value) (step S345/Yes), the controlsection 10 recommends purchase of a corresponding item to the user (stepS348). In a case where a purchase probability is equal to or larger thana threshold value, because there is a high possibility that an item isto be purchased (replenished) soon, from the viewpoint of a purchasecycle, the pet robot 1 promotes the purchase of the item by voice ordisplay. Specifically, for example, on a creation screen of a purchasemaster list that is used by the user, the control section 10 presents,as purchase recommendation items, items having purchase probabilitiesthat are equal to or larger than the threshold value, and automaticallyadds the items to the purchase master list. In a case where arecommendation item is unnecessary, the user issues a deletioninstruction of the item on the creation screen of a purchase masterlist.

3-3. Third Embodiment

In the above-described second embodiment, processing of recommending apurchase item on the basis of learning has been described, but if theaccuracy of the learning increases, not only recommendation isperformed, but also a purchase master list can be generated on the basisof a learning result, and purchase can be automatically performed.Hereinafter, specific description will be given with reference to FIG.24.

FIG. 24 is a flow chart illustrating operation processing according tothe third embodiment. As illustrated in FIG. 24, first of all, thepurchase master list acquisition section 101 of the pet robot 1automatically generates a purchase master list (step S403).Specifically, the purchase master list acquisition section 101automatically generates a purchase master list by including, in thepurchase master list, items having purchase probabilities that are equalto or larger than a threshold value, on the basis of a learning resultindicated in the second embodiment. For example, the purchase masterlist acquisition section 101 automatically generates a purchase masterlist by including, in the purchase master list, items having higherpurchase probabilities, using a second threshold value higher than thefirst threshold value used in step S345 of FIG. 23.

Next, in steps S406 to S415, processing similar to steps S106 to S115described in FIG. 9 is performed. In other words, the control section 10generates an online shop item list and a physical shop item list, and onthe basis of these, generates a purchase execution list that decideswhich item is to be finally purchased from which shop (steps S406 toS412). Subsequently, the control section 10 decides a movement path ofphysical shops registered in the purchase execution list, using themovement path decision section 104 (step S415).

Subsequently, the pet robot 1 autonomously moves to a physical shop inaccordance with the decided movement path (step S418), and acquires apurchase item by performing object recognition in the physical shop(step S421). Specifically, for example, the pet robot 1 informs a shopstaff of a purchase item by voice or display, and asks the shop staff toput the purchase item into a shopping basket or cart. In addition, thepet robot 1 including a structure for seizing an object seizes apurchase item on the basis of an object recognition result and puts thepurchase item into a basket or a cart. As described above, for example,in the case of a bipedal walking humanoid agent robot, an operation ofseizing a commercial product and putting the commercial product into ashopping cart using an arm and fingers of an upper half of the body isalso enabled.

Next, the pet robot 1 moves to a checkout counter and pays a bill of thepurchase item (step S424). For example, paying processing is manuallyperformed by a shop staff, or automatically performed.

Subsequently, in the purchase execution list, the purchase executionlist generation section 103 shifts a status of an item of which purchasehas been completed, to the “purchase completion” state (step S427).

Steps S418 to S427 described above are repeated until the pet robot 1moves to all the physical shops registered in the purchase executionlist (step S430).

Next, the online purchase processing section 110 performs processing ofcollectively purchasing items left in the purchase execution list, foreach online shop (step S433).

Subsequently, in the purchase execution list, the purchase executionlist generation section 103 shifts a status of an item of which purchasehas been completed, to the “purchase completion” state (step S436).

Then, the control section 10 of the pet robot 1 updates the purchasemaster list in accordance with a change in status of the purchaseexecution list (step S439). For example, the control section 10 deletesan item of which purchase has been completed, from the purchase masterlist.

As described above, according to the third embodiment, it becomespossible for the pet robot 1 to automatically perform all the processingfrom the decision of a purchase item to purchase.

3-4. Fourth Embodiment

The above-described third embodiment uses a completely-automatic mode,but the present disclosure is not limited to this, and in a case where asituation different from usual, such as a change in body temperature ofthe user, is caused (e.g. a case where the user is tired, a case wherethe user feels bad, etc.), the mode may be shifted to a recommendationmode, and automatic purchase may be executed as necessary. Hereinafter,specific description will be given with reference to FIG. 25.

FIG. 25 is a flow chart illustrating operation processing according tothe fourth embodiment. As illustrated in FIG. 25, first of all, the petrobot 1 measures body temperature of the user (step S453). For example,the pet robot 1 detects the body temperature of the user using a thermalsensor or a thermography. In a case where the pet robot 1 is providedwith these sensors, the pet robot 1 can detect the body temperature ofthe user when the user touches the pet robot 1.

Next, in a case where the body temperature is equal to or larger than athreshold value (e.g. 37.0 degrees Celsius) (step S456/Yes), the controlsection 10 acquires a recommendation list of items effective when feveris caused (step S459). The recommendation list may be prestored in thestorage section 20, or may be acquired from a predetermined server on anetwork. The recommendation list includes commercial products necessarywhen fever is caused. In addition, the control section 10 may askpredetermined artificial intelligence about “what is a commercialproduct necessary when fever is caused” or the like, and use the resultas a recommendation list. In addition, a determination criterion of thecontrol section 10 is not limited to a determination criterion as towhether or not the body temperature is equal to or larger than athreshold value, and for example, whether or not the body temperature ofthe user is larger than a measurement average value by a predeterminedtemperature or more may be used as a determination criterion.

Subsequently, the control section 10 presents the recommendation list tothe user from the display section 21, for example, and promotes theselection of a purchase item (step S462).

Next, in a case where the user has performed selection (step S465/Yes),the control section 10 sets a delivery date d1 on which a commercialproduct selected by the user is necessary, to several hours, forexample, and adds the delivery date d1 to a purchase master list (stepS468).

In this manner, the control section 10 can generate a purchase masterlist considering the selection of the user. Then, after a purchasemaster list is generated, similarly to the operation processingillustrated in FIG. 24, it becomes possible for the pet robot 1 toautomatically purchase an item in a physical shop or an online shop, anddeliver a necessary item to the user feeling bad. Note that, in theexample described above, “body temperature” is detected as an example ofdetermining a change in situation of the user, but the presentembodiment is not limited to this, and for example, a change from usualin user situation may be determined by detecting heartbeat, an amount ofperspiration, brain waves, an amount of activity, or the like of theuser. For example, on the basis of acceleration sensor informationdetected by an acceleration sensor provided on the user, the pet robot 1can detect a motion of the user, compare the detected motion with ausual amount of activity, and determine a change from usual in usersituation such as whether or not the user is tired or whether or not theuser feels bad.

«4. Supplement»

In the embodiments described above, the pet robot 1 independentlyperforms recommendation of a purchase shop of an item, generation of apurchase master list, and automatic purchase, but the present embodimentis not limited to this, and the above-described processing can also beexecuted on the server 3 side, and the pet robot 1 can be controlled bythe server 3.

As illustrated in FIG. 2, for example, the pet robot 1 can connect tothe network 2 in a wired/wireless manner, and perform data communicationwith the server 3 on the network. The pet robot 1 transmits collecteduser speech voice, a captured image that has been captured, currentposition information, and the like to the server 3, and recommendationof a purchase shop of an item, generation of a purchase master list, andautomatic purchase control can be performed by the server 3. Inaccordance with the control of the server 3, the pet robot 1 can performinformation presentation to the user and a movement to a physical shop.For example, the pet robot 1 receives a purchase execution list from theserver 3, and on the basis of the purchase execution list, outputs, tothe user, by voice or display, information for recommending purchase ina physical shop for partial commercial products, and recommendingpurchase in an online shop for the other commercial products.

Here, a configuration example of the server 3 will be specificallydescribed with reference to FIG. 26. FIG. 26 is a block diagramillustrating an example of a configuration of the server 3 according tothe present embodiment. As illustrated in FIG. 26, the server 3 includesa control section 30, a communication section 31, the physical shop DB32, the online shop DB 33, a purchase master list DB 34, an online shoplist DB 35, a physical shop list DB 36, an online shop item list DB 37,a physical shop item list DB 38, and a purchase execution list DB 39.

The communication section 31 is a communication module for performingtransmission and reception of data with another device. For example, thecommunication section 31 connects with the pet robot 1 via the network2, and performs transmission and reception of data.

The control section 30 functions as an arithmetic processing unit and acontrol device, and controls the entire operations performed inside theserver 3, in accordance with various types of programs. The controlsection 30 is implemented by an electronic circuit such as a CPU or amicroprocessor, for example. In addition, the control section 30 mayinclude a ROM that stores programs, calculation parameters, and the likethat are to be used, and a RAM that temporarily stores appropriatelyvarying parameters and the like.

In addition, the control section 30 according to the present embodimentfunctions as a purchase master list acquisition section 301, a shop listgeneration section 302, a purchase execution list generation section303, a movement path decision section 304, a drive control section 305,a display control section 306, a voice output control section 307, auser manipulation recognition section 308, a voice recognition section309, an online purchase processing section 310, an object recognitionsection 311, and a learning section 312. Note that the functions of therespective constituent elements are similar to correspondingconfigurations of the control section 10 of the pet robot 1 that havebeen described with reference to FIG. 4.

In this manner, the information processing system according to thepresent embodiment may employ a configuration in which the server 3performs main processing. Note that the configuration of the server 3 isnot limited to the example illustrated in FIG. 26, and a configurationin which a part of the functions of the control section 10 of the petrobot 1 is performed by the server 3 may be employed.

«5. Conclusion»

As described above, in an information processing system according to anembodiment of the present disclosure, optimum shopping in a physicalshop and an online shop can be supported.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

For example, computer programs for executing the functions of the petrobot 1 or the server 3 can be created in pieces of hardware such as aCPU, a ROM, and a RAM that are built in the pet robot 1 or the server 3of the above-described information processing system. In addition, acomputer readable storage medium storing the computer programs is alsoprovided.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An agent robot control system including:

an acquisition section configured to acquire a purchase master listindicating a candidate of a purchase commercial product of a user; and

a control section configured to generate a purchase execution list forrecommending purchase in a physical shop for a partial commercialproduct of the purchase master list, and recommending purchase in anonline shop for another commercial product.

(2)

The agent robot control system according to (1), in which the controlsection

compares a price in an online shop and a price in a physical shop ofeach commercial product in the purchase master list, and

generates, on the basis of a comparison result, a purchase executionlist for recommending purchase in a physical shop for a partialcommercial product of the purchase master list, and recommendingpurchase in an online shop for another commercial product.

(3)

The agent robot control system according to (2), in which the physicalshop is a physical shop located within a predetermined range from a homeof a user.

(4)

The agent robot control system according to (1) to (3), in which thecontrol section generates, on the basis of the purchase execution list,a movement path from a current position of an agent robot to a physicalshop in the purchase execution list.

(5)

The agent robot control system according to (4), in which, when thecontrol section detects a commercial product of which purchase has beencompleted in the physical shop, via the agent robot that has moved tothe physical shop on the basis of the movement path,

the control section performs control so as to perform a purchaseprocedure in the online shop of a commercial product in the purchasemaster list that is other than the commercial product of which purchasehas been completed.

(6)

The agent robot control system according to (4) or (5), in which, uponarrival at the physical shop, the control section performs control so asto present, by display or voice, a commercial product in the purchasemaster list that is recommended to be purchased in the physical shop.

(7)

The agent robot control system according to any one of (4) to (6), inwhich the agent robot control system is an agent robot,

a drive section configured to cause the agent robot to autonomously moveis included, and

the control section moves the agent robot to a physical shop bycontrolling the drive section on the basis of the movement path.

(8)

The agent robot control system according to (7), in which, when aprobability of purchasing a commercial product is estimated to be equalto or larger than a first value, the control section of the agent robotrecommends the user to purchase the commercial product, andautomatically adds the commercial product to the purchase master list asa purchase commercial product candidate.

(9)

The agent robot control system according to (7), in which, when aprobability of purchasing a commercial product is estimated to be equalto or larger than a second value larger than a first value, the controlsection generates a control signal for controlling the agent robot toautomatically purchase the commercial product.

(10)

The agent robot control system according to (8) or (9), in which apurchase probability of the commercial product is calculated on thebasis of a replenishment history of a commercial product.

(11)

The agent robot control system according to any one of (1) to (7), inwhich, when the control section recognizes that a state of a user thathas been detected via an agent robot has an abnormality,

the control section

acquires a purchase recommendation list corresponding to theabnormality, and presents the purchase recommendation list to a user viathe agent robot, and

generates a control signal for adding a commercial product selected bythe user from among the purchase recommendation list, to a purchasemaster list.

(12)

An agent robot system including:

a server including

a first communication section configured to acquire a purchase masterlist indicating a candidate of a purchase commercial product of a user,and

a first control section configured to generate a purchase execution listfor recommending purchase in a physical shop for a partial commercialproduct of the purchase master list, and recommending purchase in anonline shop for another commercial product, and perform control so as totransmit the purchase execution list to an agent robot via the firstcommunication section; and

an agent robot including

a second communication section configured to receive the purchaseexecution list,

an output section configured to output information to a user, and

a control section configured to control the output section to output, tothe user, information recommending purchase in a physical shop for thepartial commercial product, and recommending purchase in an online shopfor the other commercial product, on the basis of the purchase executionlist.

(13)

An agent robot control method including:

by a processor,

acquiring a purchase master list indicating a candidate of a purchasecommercial product of a user; and

generating a purchase execution list for recommending purchase in aphysical shop for a partial commercial product of the purchase masterlist, and recommending purchase in an online shop for another commercialproduct.

(14)

A storage medium having an agent robot control program stored thereon,the agent robot control program causing a computer to function as:

an acquisition section configured to acquire a purchase master listindicating a candidate of a purchase commercial product of a user; and

a control section configured to generate a purchase execution list forrecommending purchase in a physical shop for a partial commercialproduct of the purchase master list, and recommending purchase in anonline shop for another commercial product.

REFERENCE SIGNS LIST

1 pet robot

2 network

3 server

4 (4A to 4D) physical shop

5 online shop

6 trolley

10 control section

11 communication section

14 position information acquisition section

15 camera

16 voice input section

17 biosensor

18 drive section

19 touch sensor

20 storage section

21 display section

22 voice output section

23 acceleration sensor

24 angular velocity sensor

30 control section

31 communication section

32 physical shop DB

33 online shop DB

34 purchase master list DB

35 online shop list DB

36 physical shop list DB

37 online shop item list DB

38 physical shop item list DB

39 purchase execution list DB

101 purchase master list acquisition section

102 shop list generation section

103 purchase execution list generation section

104 movement path decision section

105 drive control section

106 display control section

107 voice output control section

108 user manipulation recognition section

109 voice recognition section

110 online purchase processing section

111 object recognition section

112 learning section

200 body portion unit

201 leg portion unit

202 head portion unit

203 tail portion unit

210 purchase master list DB

211 purchase master list

220 online shop list DB

221 online shop list

230 physical shop list DB

231 physical shop list

240 online shop item list DB

241 online shop item list

250 physical shop item list DB

251 physical shop item list

260 purchase execution list DB

261 purchase execution list

271 current in-refrigerator item list

272 previous in-refrigerator item list

273 difference list

274 purchase cycle probability list

301 purchase master list acquisition section

302 shop list generation section

303 purchase execution list generation section

304 movement path decision section

305 drive control section

306 display control section

307 voice output control section

308 user manipulation recognition section

309 voice recognition section

310 online purchase processing section

311 object recognition section

312 learning section

1. An agent robot control system, comprising: circuitry configured to:acquire a purchase master list that includes a first plurality ofcommercial products corresponding to purchase candidate items of a user,wherein the first plurality of commercial products includes a firstpartial set and a second partial set different from the first partialset; generate a purchase execution list to recommend purchase in aphysical shop for the first partial set and purchase in an online shopfor the second partial set; and generate, based on the generatedpurchase execution list, a movement path from a current position of anagent robot associated with the agent robot control system to thephysical shop, wherein the agent robot includes a drive control sectionthat controls the agent robot to move the agent robot from the currentposition of the agent robot to the physical shop, and the controlsection controls the agent robot based on the movement path.
 2. Theagent robot control system according to claim 1, wherein the circuitryis further configured to: compare a price in the online shop and a pricein the physical shop of each of the first plurality of commercialproducts, and generate, based on the comparison, the purchase executionlist to recommend the purchase in the physical shop for the firstpartial set and the purchase in the online shop for the second partialset.
 3. The agent robot control system according to claim 2, wherein thephysical shop is a shop located within a specific range from a home ofthe user.
 4. The agent robot control system according to claim 1,wherein upon arrival of the agent robot at the physical shop, thecircuitry is further configured to present, by one of display or voice,the first partial set.
 5. The agent robot control system according toclaim 1, wherein the agent robot control system is the agent robot, thecircuitry is further configured to control the agent robot to move theagent robot from the current position to the physical shop.
 6. The agentrobot control system according to claim 5, wherein, based on aprobability of purchasing each of a second plurality of commercialproducts being equal to or larger than a value, the circuity is furtherconfigured to: control the agent robot to recommend the user to purchasethe second plurality of commercial products; and automatically add thesecond plurality of commercial products to the purchase master list asthe purchase candidate items.
 7. The agent robot control systemaccording to claim 6, wherein the probability of purchasing each of thesecond plurality of commercial products is calculated based on areplacement history of each of the second plurality of commercialproducts.
 8. The agent robot control system according to claim 5,wherein, based on a probability of purchasing each of a second pluralityof commercial products being equal to or larger than a second value thatis larger than a first value, the control section is further configuredto generate a control signal to control the agent robot to automaticallypurchase the second plurality of commercial products.
 9. The agent robotcontrol system according to claim 1, wherein the circuitry is furtherconfigured to recognize abnormality of a state of the user, the state ofthe user is detected by the agent robot, based on the state of the userthat has the abnormality, the circuitry is further configured to:acquire a purchase recommendation list that corresponds to theabnormality of the user; present the purchase recommendation list to theuser via the agent robot; and generate a control signal to add acommercial product selected by the user from the purchase recommendationlist, and the commercial product selected from the purchaserecommendation list is added to the purchase master list.
 10. An agentrobot control method, comprising: in an agent robot control system:acquiring a purchase master list that includes a first plurality ofcommercial products corresponding to purchase candidate items of a user,wherein the first plurality of commercial products includes a firstpartial set and a second partial set different from the first partialset; generating a purchase execution list to recommend purchase in aphysical shop for the first partial set and purchase in an online shopfor the second partial set; and generating, based on the generatedpurchase execution list, a movement path from a current position of anagent robot associated with the agent robot control system to thephysical shop, wherein the agent robot includes a drive control sectionthat controls the agent robot to move the agent robot from the currentposition of the agent robot to the physical shop, and the controlsection controls the agent robot based on the movement path.
 11. Theagent robot control method according to claim 10, further comprising:comparing a price in the online shop and a price in the physical shop ofeach of the first plurality of commercial products, and generating,based on the comparison, the purchase execution list to recommend thepurchase in the physical shop for the first partial set and the purchasein the online shop for the second partial set.
 12. The agent robotcontrol method according to claim 11, wherein physical shop is a shoplocated within a specific range from a home of the user.
 13. The agentrobot control method according to claim 10, further comprisingpresenting, by one of display or voice, the first partial set uponarrival of the agent robot at the physical shop.
 14. The agent robotcontrol method according to claim 10, further comprising controlling theagent robot to move the agent robot from the current position to thephysical shop, wherein the agent robot control system is the agentrobot.
 15. The agent robot control method according to claim 14, whereinbased on a probability of purchasing each of a second plurality ofcommercial products being equal to or larger than a value, the methodfurther comprising: controlling the agent robot to recommend the user topurchase the second plurality of commercial products; and automaticallyadding the second plurality of commercial products to the purchasemaster list as the purchase candidate items.
 16. The agent robot controlmethod according to claim 15, wherein the probability of purchasing eachof the second plurality of commercial products is calculated based on areplacement history of each of the second plurality of commercialproducts.
 17. The agent robot control method according to claim 14,wherein based on a probability of purchasing each of a second pluralityof commercial products being equal to or larger than a second value thatis larger than a first value, the method further comprising: generatinga control signal to control the agent robot to automatically purchasethe second plurality of commercial products.
 18. The agent robot controlmethod according to claim 10, further comprising: recognizingabnormality of a state of the user, wherein the state of the user isdetected by the agent robot, based on the state of the user that has theabnormality, the method further comprising: acquiring a purchaserecommendation list that corresponds to the abnormality of the user;presenting the purchase recommendation list to the user via the agentrobot; and generating a control signal to add a commercial productselected by the user from the purchase recommendation list, and thecommercial product selected from the purchase recommendation list isadded to the purchase master list.